מקורות לחומר בשיעור ספר הלימוד: Forsyth & Ponce מאמרים שונים חומר באינטרנט! פרק פרק 18

Transcription

1 עקיבה

2 מקורות לחומר בשיעור ספר הלימוד: פרק 5..2 Forsh & once פרק 8 מאמרים שונים חומר באינטרנט!

3 Toda Tracking wih Dnamics Deecion vs. Tracking Tracking as probabilisic inference redicion and Correcion Linear Dnamic Models The Kalman Filer Kalman filer for D sae General Kalman filer Limiaions

4 Deecion vs. Tracking = =2 =2 =2

5 Deecion vs. Tracking = =2 =2 =2 Deecion We deec he objec independenl in each frame and can record is posiion over ime e.g. based on blob s cenroid or deecion window coordinaes.

6 Deecion vs. Tracking = =2 =2 =2 Tracking wih dnamics: We use image measuremens o esimae he objec posiion bu also incorporae he posiion prediced b dnamics i.e. our expecaion of he objec s moion paern.

7 Tracking wih Dnamics Ke idea Given a model of expeced moion predic where objecs will occur in nex frame even before seeing he image. In nex frame updae predicion using acual measuremens

8 Illusraion iniial posiion predicion measuremen updae x x x x

9 Tracking wih Dnamics Ke idea Given a model of expeced moion predic where objecs will occur in nex frame even before seeing he image. In nex frame updae predicion using acual measuremens Goals Resric search for he objec Improved esimaes since measuremen noise is reduced b rajecor smoohness. Assumpion: coninuous moion paerns Camera is no moving insanl o new viewpoin. Objecs do no disappear and reappear in differen places. Gradual change in pose beween camera and scene.

10 General Model for Tracking sae x sae x 2 sae x 3 sae x 4 measuremen 2 3 4

11 General Model for Tracking The moving objec of ineres is characerized b an underling sae Sae gives rise o measuremens or observaions Y A each ime he sae changes o and we ge a new observaion Y 2 Y Y 2 Y

12 Sae vs. Observaion Hidden sae : parameers of ineres (e.g. locaion of poin conour of shape ec.) Measuremen : wha we ge o direcl observe (e.g. pixel values image feaures)

13 Tracking as Inference Our goal: recover mos likel sae given All observaions seen so far. Knowledge abou dnamics of sae ransiions. In oher words maximize Differen approaches include: HMMs Kalman filers Condensaion p( x )...

14 Seps of racking redicion: Wha is he nex sae of he objec given pas measuremens? Y Y

15 Seps of racking redicion: Wha is he nex sae of he objec given pas measuremens? Correcion: Compue an updaed esimae of he sae from predicion and measuremens Y Y Y Y Y

16 Seps of racking redicion: Wha is he nex sae of he objec given pas measuremens? Correcion: Compue an updaed esimae of he sae from predicion and measuremens Y Y Y Y Y

17 Simplifing assumpions Onl he immediae pas maers dnamics model

18 Simplifing assumpions Onl he immediae pas maers Measuremens depend onl on he curren sae Y Y Y Y dnamics model observaion model

19 Simplifing assumpions Onl he immediae pas maers Measuremens depend onl on he curren sae Y Y Y Y dnamics model observaion model 2 - Y Y 2 Y - Y

20 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Tracking as Inducion Base case: Assume we have iniial prior ha predics sae in absence of an evidence: ( ) A he firs frame correc his given he value of Y = ( ) ( ) ( Y ) ( ) ( ) ( ) oserior prob. of sae given measuremen Likelihood of measuremen rior of he sae

21 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Tracking as Inducion Base case: Assume we have iniial prior ha predics sae in absence of an evidence: ( ) A he firs frame correc his given he value of Y = Given correced esimae for frame : redic for frame + Correc for frame + predic correc

22 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Inducion Sep: redicion redicion involves represening given d d d Law of oal probabili A A B db

23 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Inducion Sep: redicion redicion involves represening given d d d Condiioning on A B A B B

24 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Inducion Sep: redicion redicion involves represening given d d d Independence assumpion

25 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Inducion Sep: Correcion Correcion involves compuing given prediced value d Baes rule B A A A B B

26 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Inducion Sep: Correcion Correcion involves compuing given prediced value d Independence assumpion (observaion depends onl on sae )

27 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Inducion Sep: Correcion Correcion involves compuing given prediced value d Condiioning on

28 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Inducion Sep: Correcion Correcion involves compuing given prediced value d observaion model prediced esimae normalizaion facor

29 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Summar: redicion and Correcion redicion: d Dnamics model Correced esimae from previous sep

30 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Summar: redicion and Correcion redicion: Correcion: 3 B. Leibe d Dnamics model Correced esimae from previous sep d Observaion model rediced esimae

31 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Topics of This Lecure Tracking wih Dnamics Deecion vs. Tracking Tracking as probabilisic inference redicion and Correcion Linear Dnamic Models The Kalman Filer Kalman filer for D sae General Kalman filer Limiaions B. Leibe 32

32 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Noaion Reminder x ~ N( μ Σ) Random variable wih Gaussian probabili disribuion ha has he mean vecor μ and covariance marix Σ. x and μ are d-dimensional Σ is d x d. d=2 d= If x is D we jus have one Σ parameer: he variance σ 2 B. Leibe 33

33 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Linear Dnamic Models Dnamics model Sae undergoes linear ranformaion D plus Gaussian noise x ~ N D x d n nn n Observaion model Measuremen is linearl ransformed sae plus Gaussian noise ~ N M x m m mn n B. Leibe 34

34 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Example : Randoml Drifing oins Consider a saionar objec wih sae as posiion. osiion is consan onl moion due o random noise erm. x p p p Sae evoluion is described b ideni marix D=I x D x noise Ip noise B. Leibe 35

35 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Example 2: Consan Veloci (D oins) Measuremens Saes ime B. Leibe 36 Figure from Forsh & once

36 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Example 2: Consan Veloci (D oins) Sae vecor: posiion p and veloci v Measuremen is posiion onl ) ( v v v p p v p x noise v p noise D x x (greek leers denoe noise erms) noise v p noise Mx

37 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing 38 B. Leibe Figure from Forsh & once Example 3: Consan Accel. (D oins)

38 ercepual and Sensor Augmened Compuing Compuer Vision WS 8/9 Example 3: Consan Accel. (D oins) Sae vecor: posiion p veloci v and acceleraion a. Measuremen is posiion onl 39 ) ( ) ( a a a v v v p p a v p x noise a v p noise D x x (greek leers denoe noise erms) noise a v p noise Mx

39 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Topics of This Lecure Tracking wih Dnamics Deecion vs. Tracking Tracking as probabilisic inference redicion and Correcion Linear Dnamic Models The Kalman Filer Kalman filer for D sae General Kalman filer Limiaions B. Leibe 4

40 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing The Kalman Filer Mehod for racking linear dnamical models in Gaussian noise The prediced/correced sae disribuions are Gaussian You onl need o mainain he mean and covariance. The calculaions are eas (all he inegrals can be done in closed form). B. Leibe 4

41 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing ropagaion of Gaussian densiies

42 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing The Kalman Filer Know correced sae from previous ime sep and all measuremens up o he curren one redic disribuion over nex sae. Receive measuremen Know predicion of sae and nex measuremen Updae disribuion over curren sae. Time updae ( redic ) Measuremen updae ( Correc ) Mean and sd. dev. of prediced sae: B. Leibe Time advances: ++ Mean and sd. dev. of correced sae: 43

43 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Kalman Filer for D Sae Wan o represen and updae 2 x N ( ) 2 x N ( ) B. Leibe 44

44 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing D Kalman Filer: redicion Have linear dnamic model defining prediced sae evoluion wih noise Wan o esimae prediced disribuion for nex sae 2 N ( 2 ~ N dx d ) Updae he mean: d for derivaions see F& Chaper 8 Updae he variance: ( B. Leibe ) 2 2 d ( d ) 2 45

45 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing D Kalman Filer: Correcion Have linear model defining he mapping of sae o measuremens: 2 Y ~ N m mx Wan o esimae correced disribuion given laes measuremen: Updae he mean: Updae he variance: B. Leibe 2 N ( ) 2 m 2 m m m 2 ( ( m ( ) m m ( ) 46 ( ) ) ) 2 Derivaions: F& Chaper 8

46 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing redicion vs. Correcion m ( ) m 2 m ( ) m m ( ) m m ( ) ( ) Wha if here is no predicion uncerain 2 ( ) The measuremen is ignored! ( )? Wha if here is no measuremen uncerain ( ) m B. Leibe The predicion is ignored! 2 ( m )? 47

47 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing posiion Recall: Consan Veloci Example measuremens sae Sae is 2D: posiion + veloci Measuremen is D: posiion B. Leibe ime 48 Figure from Forsh & once

48 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Consan Veloci Model o sae x measuremen * prediced mean esimae + correced mean esimae bars: variance esimaes before and afer measuremens B. Leibe 49 Figure from Forsh & once

49 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Consan Veloci Model o sae x measuremen * prediced mean esimae + correced mean esimae bars: variance esimaes before and afer measuremens B. Leibe 5 Figure from Forsh & once

50 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Consan Veloci Model o sae x measuremen * prediced mean esimae + correced mean esimae bars: variance esimaes before and afer measuremens B. Leibe 5 Figure from Forsh & once

51 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Consan Veloci Model o sae x measuremen * prediced mean esimae + correced mean esimae bars: variance esimaes before and afer measuremens B. Leibe 52 Figure from Forsh & once

52 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Kalman Filer: General Case (>dim) Wha if sae vecors have more han one dimension? REDICT CORRECT x D x D K T M M T D x x K Mx d I KM M T m residual More weigh on residual when measuremen error covariance approaches. for derivaions see F& Chaper 8 B. Leibe Less weigh on residual as a priori esimae error covariance approaches.

53 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Summar: Kalman Filer ros: Gaussian densiies everwhere Simple updaes compac and efficien Ver esablished mehod ver well undersood Cons: Unimodal disribuion onl single hpohesis Resriced class of moions defined b linear model B. Leibe 54

54 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Wh Is This A Resricion? Man ineresing cases don have linear dnamics E.g. pedesrians walking E.g. a ball bouncing B. Leibe 55

55 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Ball Example: Wha Goes Wrong Here? Assuming consan acceleraion model redicion redicion is oo far from rue posiion o compensae ossible soluion: Exended Kalman Filer Keeps muliple differen moion models in parallel I.e. would check for bouncing a each ime sep B. Leibe redicion 2 redicion 3 redicion 4 redicion 5 56 Correc predicion

56 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Tracking issues Deciding on he srucure of he model Iniializaion Specifing he dnamics model Specifing he observaion model Daa associaion problem: which measuremens ell us abou he objec(s) being racked?

57 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Daa associaion Simple sraeg: onl pa aenion o he measuremen ha is closes o he predicion

58 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Daa associaion Simple sraeg: onl pa aenion o he measuremen ha is closes o he predicion Doesn alwas work

59 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Daa associaion Simple sraeg: onl pa aenion o he measuremen ha is closes o he predicion More sophisicaed sraeg: keep rack of muliple sae/observaion hpoheses This is a general problem in compuer vision here is no eas soluion

60 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Tracking issues Deciding on he srucure of he model Iniializaion Specifing he dnamics model Specifing he observaion model Daa associaion problem redicion vs. correcion If he dnamics model is oo srong will end up ignoring he daa If he observaion model is oo srong racking is reduced o repeaed deecion Drif

61 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing Drif D. Ramanan D. Forsh and A. Zisserman. Tracking eople b Learning heir Appearance. AMI 27.

62 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing אז מה ראינו היום?

63 Compuer ercepual Vision and Sensor WS 8/9 Augmened Compuing עקיבה כבעיית הסקה הסתברותית מודלים לינאריים לשינוי מצב מערכת Kalman filering aricle Filering סיכום

64 Good luck!